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Abstract 

Electrochemistry is found to be a difficult topic to leam due to its abstract concepts that involve three 
representation levels. Research showed that animation and simulation using Information and Communication 
Technology can help students to visualize and thus enhance students’ understanding in learning abstract 
chemistry topics. As a result, an interactive multimedia module with a pedagogical agent (IMMPA) named EC 
Lab was developed to assist students in the learning of Electrochemistry. There were 35 students and seven 
experts involved in this formative evaluation to test the usability and effectiveness of the module developed. 
Instmments involved were module evaluation questionnaire, module reliability questionnaire, module validity 
questionnaire, pretest, post-test and motivation questionnaire. Results showed that the students and experts rated 
the module developed as high (M = 3.94) and very high quality (M = 4.45). Besides, the respondents also see the 
module as reliable (M = 3.87) and valid (M = 4.51). Students have higher scores on their post-test and have 
gained higher motivation level after learning with the IMMPA EC Lab. However, some corrections and 
modifications have been made based on feedback and suggestions from the respondents to improve the quality of 
IMMPA EC Lab. 

Keywords: interactive multimedia module, pedagogical agent, electrochemistry, formative evaluation 

1. Introducation 

1.1 Learning of Electrochemistry in Malaysia 

Electrochemistry is the sixth chapter in the Malaysian Chemistry syllabus for secondary schools. This topic is 
taught to upper secondary level students in the science stream. Electrochemistry is a study of inter-conversion of 
chemical energy and electrical energy that occurs in electrolytic and voltaic cells. Previous studies (Bojczuk, 
1982; Lee & Kamisah, 2010; Lin, Yang, Chiu & Chou, 2002; Roziah, 2005) showed that the topic is difficult to 
learn because the concepts are abstract. Students often encounter misconceptions in the learning of this topic 
(Garnett & Hackling 1993; Garnett & Treagust 1992; Garnett, Garnett & Hackling, 1995; Karsli & (/’alik, 2012; 
Lee & Mohammad Yusof 2009; Lee 2008; Lin et al. 2002; Sanger & Greenbowe 1997a; Sanger & Greenbowe 
1997b). 

Macroscopically, students need to study the concepts of electrolytes and non-electrolytes, the electrolysis process 
and voltaic cells. Microscopically, they need to understand the movement of ions and electrons during the 
electrolysis process. Besides that, they need to transform the process into chemical formulae and equations 
symbolically. Students face difficulties in understanding the abstract chemical processes especially at the 
microscopic and symbolic levels (Garnett & Hackling 1993; Garnett & Treagust 1992; Garnett et al., 1995; 
Karsli & £alik, 2012; Lee & Mohammad Yusof 2009; Lee 2008; Lin et al. 2002; Sanger & Greenbowe 1997a; 
Sanger & Greenbowe 1997b). 

Studies (Doymus, Karacop & Simsek, 2010; Gois & Giordan, 2009; Lerman & Morton, 2009) showed that 
animation and simulation using information and communication technology (ICT) can help students to visualize 
and thus enhance students’ understanding in learning abstract chemistry topics. The use of multimedia creates the 
environment where students can visualize the abstract chemical processes via animation and video at 
macroscopic, microscopic and symbolic levels (Bowen, 1998; Burke, Greenbowe & Windschitl, 1998; 
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Rodrigues, Smith & Ainley, 2001; Russell, Kozma, Jones, Wykoff, Marx & Davis, 1997). 

Although the use of multimedia modules is able to assist students in visualizing the abstract concepts, students 
lack sufficient metacognitive awareness and comprehension monitoring skill to make effective choices (Hill & 
Hannafin, 2001; Land, 2000) in computer-mediated learning environments. They lack the skills to find, process 
and use information and ideas. Students as novice learners do not always make connections with prior 
knowledge or everyday experiences in ways that are productive for learning (Land, 2000). As a result. 
Pedagogical Agents (PAs) are designed to facilitate learning in computer-mediated learning environments (Chou, 
Chan & Lin, 2003; Craig, Gholson & Driscoll, 2002; Johnson, Rickel, & Lester, 2000; Moundridou & Virvou, 
2002; Predinger, Saeyor & Ishizuka, n.d., Slater, 2000). 

PAs in the interactive multimedia module serve to enhance students' metacognitive awareness of what they know 
and what they should know regarding the topic being studied. One of the strategies to provide metacognitive 
guidance involves embedding support, or scaffolds for procedural, strategic, or metacognitive control (Land, 
2000). Such guidance and support are provided by the PAs in the module. PAs could make learners more aware 
of the opportunities presented to them, provide advice to learners on the tools to be used, and explain the 
functionalities of the tools in an open learning environment (Clarebout & Elen, 2007). Hence, an Interactive 
Multimedia Module with Pedagogical Agents (IMMPA) named EC Lab was developed to assist students in the 
learning of Electrochemistry. 

1.2 Development of the IMMPA EC Lah 

In this study, we developed IMMPA EC Lab in the learning of Electrochemistry by combining two instructional 
design models: Kemp Model (1994) and Gerlach and Ely Model (1980). The two models are combined as they 
are both classroom-oriented (Gustafson & Branch, 1997) with their own strengths. The model produced through 
the combination is named KemGerly Model and the conceptual framework of KemGerly Model is illustrated in 
Figure 1. 



Figure 1. Conceptual framework of KemGerly Model 


The Kemp Model describes elements, not steps, stages, levels or sequential items in an instructional design (Kemp, 
Morrison & Ross, 2004). The oval shape of the model indicates the independence of the elements in the model. It is a 
non-linear model with no starting and ending point. All the processes of designing, developing, implementing and 
evaluating can be done concurrently and continuously. The Gerlach and Ely Model is suitable for novice instructional 
designers who have knowledge and expertise in a specific context (Qureshi, 2001, 2003, 2004). This model is 
classroom-oriented and is suitable for teachers at secondary schools and higher education institutions. The Gerlach and 
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Ely Model focuses more on instructional materials and resources without identifying instructional problems. Hence, 
we have combined the two models as the instructional design model to develop the IMMPA EC Lab. 

To develop IMMPA EC Lab, first of all, we need to identify instructional problems in Electrochemistry (Lee & 
Kamisah, 2010) and study learners’ characteristics (Lee & Kamisah, 2012). Then, the four elements in the Kemp 
Model were replaced by some steps in Gerlach & Ely Model. We analyzed the contents of Electrochemistry, specified 
the learning objectives and assessed students’ entering behavior. We also determined the strategy to be used, organized 
the group, considered the allocation of time and space and finally selected suitable resources in designing the IMMPA 
EC Lab. Next, we designed the message in the content delivery and arrange the instructional delivery. Evaluation 
instruments were also developed as the last part of every sub unit in the IMMPA EC Lab to assess students’ 
understanding in each sub unit. When everything was done, a formative evaluation was carried out to evaluate the 
module developed. 

1.2.1 Formative Evaluation 

The main purpose of the formative evaluation is to inform the instructor or planning team how well the 
instructional program has been serving the objectives as it progresses (Morrison, Ross & Kemp, 2007). Hence, 
the formative evaluation should be conducted at different stages during the development of the IMMPA EC Lab. 
Dick and Carey (1991) suggested three stages of formative evaluation when designing an instruction, namely 
one-to-one trials, small-group trials and field trials. Descriptions of the stages with the related instruction phase, 
purpose, respondents and instruments are displayed in Table 1. 


Table 1. Stages of Formative Evaluation 


Stage 

Instruction Phase 

Purpose 

Learners 

Main Measures 

One-to-one trials 

Development 

Try-out impressions 

Individuals 

Observation, survey, 
interview 

Small-group 

Preliminary/draft 

Identify 

Small groups (8-20) 

Observation, attitudes, 

trials 

version 

strengths/weakness 


performance 

Field trials 

Completed 

Assess actual 
implementation 

Regular classes 

Performance, attitudes 


As shown in Table 1, we carried out one-to-one trials during the development phase. Next, we carried out 
small-group trials followed by field trials. This paper will concentrate on details of the small-group trials to test 
whether the module developed is usable and effective in learning Electrochemistry. 

Flagg (1990) suggested four types of formative evaluation, namely connoisseur-based studies, decision-oriented 
studies, objectives-based studies and public relations-inspired studies. In our case, we decided to combine the 
connoisseur-based study approach and objective-based study approach as our method of formative evaluation. 
We selected the connoisseur-based study approach because it employs subject matter experts and other 
appropriate consultants (media and design experts) to examine the instruction and give opinions regarding its 
accuracy and effectiveness. The IMMPA EC Lab was examined by some experts in Electrochemistry, language 
experts and instructional designers to maintain the quality of IMMPA EC Lab in terms of content and 
instructional design. In addition, we administered pre- and post achievement tests and motivation questionnaires 
to measure the amount of gain on measures of achievement and attitude as the objectives-based study approach. 

1.3 Research Questions and Hypotheses 

The study was carried out as a formative evaluation for the development of IMMPA EC Lab. This formative 
evaluation is a combination of connoisseur-based study and objectives-based study approaches. The research 
questions and hypotheses that drive this formative evaluation are as follows:- 

1) What is the quality level of IMMPA EC Lab from students’ point of view? 

2) What is the quality level of IMMPA EC Lab from experts’ point of view? 

3) Is IMMPA EC Lab a reliable multimedia module in the learning of Electrochemistry? 

4) Is IMMPA EC Lab a valid multimedia module in the learning of Electrochemistry? 

5) HoE There is no significant difference between the mean scores in achievement test before and after the 
intervention. 

6) H, i2 : There is no significant difference between the mean scores in motivation before and after the 
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intervention. 

2. Method 

2.1 Respondents 

Seven experts from different areas (content, chemistry education, instructional design/e-learning) were selected 
as respondents for the connoisseur-based study. They evaluated IMMPA EC Lab in terms of pedagogical content 
of Electrochemistry, language, instructional design and validity of the module. 

Thirty-five Form Four students were selected as respondents for the objectives-based study to investigate the 
effect of IMMPA EC Lab on students’ knowledge and motivation in learning Electrochemistry. Besides that, they 
were asked to evaluate the usability of IMMPA EC Lab in terms of quality and reliability of the module. The 
respondents consisted of 13 male and 22 female students (aged 16 years), and the majority of them are Malays. 
As for small-group trials, there were 8-20 respondents (Table 1). Hence, it is argued that 35 students in our study 
are more than enough for the objective-based study. 

2.2 Materials 

Materials utilized in the study are the IMMPA EC Lab, module evaluation questionnaire, module reliability 
questionnaire, module validity questionnaire, pretest, post-test and motivation questionnaire. 

2.2.1 IMMPA EC Lab 

IMMPA EC Lab was developed using the KemGerly Model. There are two PAs in the IMMPA EC Lab, namely 
Professor T and Lisa. Professor T is a sixty-year-old male PA who acts as an expert in Electrochemistry. He gives 
accurate information and explains new concepts to the students. Professor T speaks slowly in a formal way with 
little body gestures and facial expressions. On the other hand, Lisa is a fifteen-year-old female who speaks with 
an energetic voice. She is a learning companion in the IMMPA EC Lab. She learns together with the students, as 
well as motivates and encourages the students to complete the tasks and exercises in the module. Students are 
free to choose the PA they want to accompany them in learning Electrochemistry when using the IMMPA EC 
Lab. 

The main menu for the IMMPA EC Lab consists of tutorials, experiments, exercises, quizzes, memos and games. 
There are five sub units in the IMMPA EC Lab: (1) Electrolytes and Non-Electrolytes, (2) Electrolysis of Molten 
Compounds, (3) Electrolysis of Aqueous Solutions, (4) Voltaic Cells and (5) Types of Voltaic Cells. The 
complete flow of each sub unit follows the five phases in the learning process created by Needham (1987). The 
five phases are orientation, elicitation of ideas, restructuring of ideas, application of ideas and review. The 
relationship between sessions in IMMPA EC Lab and Needham phases is summarized in Table 2 below. 


Table 2. Needham Phases and Sessions in IMMPA EC Lab 


Needham Phases 

Sessions in every Sub unit 

Main Menu 

Orientation 

Think about it! 


Elicitation of Ideas 

Restructuring of Ideas 

Do you still remember? 


• Clarification and exchange 

Give me your ideas... 

Tutorial 

• Exposure to conflict situation 

Are you sure? 


• Construction of new ideas and evaluation 

Let’s do it!! / Show time! 

Experiment 

Application of Ideas 

Practice makes perfect! 

Exercise 

Review 

Before & after... 



Test yourself 

Challenge yourself 

Quiz 

Memo 

Game 


All the information delivery for the sub units is presented in the tutorial session. The experiment session consists 
of five experiments. After the information delivery process, the students will attempt some exercises to enhance 
their understanding on the concepts learnt. A quiz is given at the end of every sub unit. A memo is created to give 
some notes on Electrochemistry concepts. There are four activities in the game session to let the students relax 
their mind after the learning process. 


53 




www.ccsenet.org/ies 


International Education Studies 


Vol. 5, No. 6; 2012 


2.2.2 Module Evaluation Questionnaire 

A module evaluation questionnaire was used to evaluate the quality of the module based on some components in 
the module. The components and number of items in the module evaluation questionnaire for both student and 
expert versions are summarized in Table 3. The items for PAs’ evaluation components were created according to 
the agents’ characteristics in Agent Persona Instrument (API) by Baylor and Ryu (2003). We referred to previous 
researches (Baylor, Ryu & Shen, 2003; Brunken, Plass, & Leutner, 2004; Chandler & Sweller, 1992; Craig et al., 
2002; Kirk, 2008; Mayer & Moreno, 1998; Mayer, Dow & Mayer, 2003; Mayer, Fennell, Farmer & Campbell, 
2004; Mayer, Sobko & Mautone, 2003; Moreno & Mayer, 2000; Moreno & Mayer, 1999; Moreno et al., 2000; 
Mousavi, Low & Sweller, 1995; Paas & Van Merrienboer, 1994; Paas, Renkl & Sweller, 2003; Pollock, Chandler 
& Sweller, 2002; Sweller & Chandler, 1994; Van Gerven, Paas, Van Merrienboer, & Schmidt, 2002; Van 
Merrienboer, Kirschner & Kester, 2003) to create items about evaluation of Cognitive Load Theory and 
Cognitive Theory of Multimedia Learning. 


Table 3. Components in Module Evaluation Questionnaire 




Components 

Total Items 

Students 

Experts 

I. 

Screen design 




A. 

Text 

4 

4 


B. 

Colour 

4 

4 


C. 

Graphic 

6 

6 


D. 

Animation 

3 

3 


E. 

Video 

3 

3 


F. 

Audio 

7 

7 

II. 

Navigation & Interactivity 

13 

13 

III. 

Language 

7 

7 

IV. 

Content 


11 

11 

V. 

Interest & Motivation 

10 

10 

VI. 

Pedagogical agent 



a) 

Professor T 




A. 

Facilitating Learning 

10 

10 


B. 

Credible 

5 

5 


C. 

Human-like 

5 

5 


D. 

Engaging 

5 

5 

b) 

Lisa 





A. 

Facilitating Learning 

10 

10 


B. 

Credible 

5 

5 


C. 

Human-like 

5 

5 


D. 

Engaging 

5 

5 

VII. 

Application of Theory 



a) 

Behaviourism Theory 

- 

9 

b) 

Cognitivism Theory 

- 

8 

c) 

Constructivism Theory 

- 

12 

d) 

Cognitive Load Theory 




A. 

Intrinsic cognitive load 

- 

8 


B. 

Extraneous cognitive load 





Split attention effect 

- 

8 



Worked-example effect 

- 

9 



Modality effect 

- 

9 



Redundancy effect 

- 

5 


C. 

Germane cognitive load 

- 

7 

e) 

Cognitive Theory of Multimedia Learning 




A. 

Personalization principle 

- 

7 


B. 

Modality principle 

- 

9 


C. 

Spatial contiguity principle 

- 

7 


D. 

Temporal contiguity principle 

- 

7 


E. 

Redundancy principle 

- 

5 


F. 

Voice principle 

- 

8 


G. 

Interactivity principle 

- 

11 

Total 



118 

247 
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The questionnaire was given to both students and experts to compare the quality of module from their point of 
view. Similar questionnaires were given to the students and experts with some addition of components in the 
experts’ version. 

2.2.3 Module Reliability Questionnaire 

A module reliability questionnaire was developed by the researchers with reference to the examples suggested by 
Sidek and Jamaludin (2005). Items in the module reliability questionnaire were developed based on activities in 
the IMMPA EC Lab because they are better compared to items created based on objectives of the module (Sidek 
& Jamaludin, 2005). The distribution of items based on activities in IMMPA EC Lab is summarized in Table 4. 


Table 4. Item Distribution in Module Reliability Questionnaire 


Sessions in every Sub unit 

Total items 

Front page of sub units 

3 

Objectives 

4 

Think about it! 

6 

Do you still remember? 

4 

Give me your ideas... 

4 

Are you sure? 

4 

Let’s do it!! / Show time! 

7 

Practice makes perfect! 

6 

Before & after... 

4 

Test yourself! 

4 

Challenge yourself! 

4 

Total 

50 


2.2.4 Module Validity Questionnaire 

We referred to Sidek’s and Jamaludin’s suggestions and developed the items in our module validity questionnaire 
based on characteristics of module validity suggested by Russell (1974). All the characteristics required and the 
items related are summarized in Table 5. 


Table 5. Item Distribution in Module Validity Questionnaire 

Characteristics of Module Validity Total items 

Accurate target population 5 

The teaching situation or implementation method is done very well 5 

Enough time to implement the whole module 5 

Potential to increase students’ academic achievement 5 

Potential to change students’ behavior to be excellent in life 5 

Total 25 


2.2.5 Achievement Tests 

Achievement tests were administered in the form of pretest and post-test before and after the intervention. There 
are two structured questions in the achievement tests. The questions test knowledge on electrolytic cell and 
voltaic cell concepts at the macroscopic, microscopic and symbolic levels. Questions in the pretest and post-test 
are similar in terms of difficulty level and concept tested. The concepts tested and numbers of items in the 
achievement tests are summarized in Table 6. 


55 




www.ccsenet.org/ies 


International Education Studies 


Vol. 5, No. 6; 2012 


Table 6. Distribution of Items in Achievement Tests 



Concept 

Item Distribution 

Representation 


Electrolytic Voltaic Cell 

Cell 

Level 


Flow of current in the conductors and in the 
electrolytes 

c(i),c(ii) 

c (i), c (ii) 

Microscopic 

Identify the anode and cathode 


k, 1 (i), 1 (ii), m (i), m 

(ii) 

Macroscopic 

Microscopic 

Identify the process happening at the anode and 
cathode 

a (i), a (ii), 
b 

a (i), a (ii), b 

Macroscopic 

Oxidation and reduction process 

e (i), e (ii), 
f 

e (i), e (ii), f 

Microscopic 

Symbolic 

Concept of Electrolyte 

j 

j 

Microscopic 


2.2.6 Motivation Questionnaire 

The motivation questionnaire is a Likert scale questionnaire. The distribution of items in the motivation 
questionnaire is shown in Table 7. 

Table 7. Item Distribution in Motivation Questionnaire 


Sub dimension 

Item Distribution 

Intrinsic goal orientation 

1, 13, 19,21 

Extrinsic goal orientation 

6, 9, 27 

Task value 

8, 14,20, 23,24 

Control of learning belief 

2, 7, 15, 22 

Self-Efficacy for learning and performance 

4,5, 10, 12, 17, 18, 26, 28 

Test anxiety 

3, 11, 16, 25 


As highlighted in Table 7, there are six constructs involved, namely intrinsic goal orientation, extrinsic goal 
orientation, task value, control of learning belief construct, self-efficacy for learning and performance and test 
anxiety. There is a total of 28 items in the questionnaire with the Likert scale ranging from 1 - Strongly Disagree 
to 5 - Strongly Agree. Items in the questionnaire have been taken from the study of Sadiah and colleagues (2009) 
which were translated from the instrument used by Pintrich and DeGroot (1990). In this study, the researchers 
used the motivation section only and changed the scale from seven points to five points. 

2.3 Procedure 

For the connoisseur-based study, the experts were given IMMPA EC Lab, a module evaluation questionnaire and 
a module validity questionnaire. They need to explore the IMMPA EC Lab and then answer the module 
evaluation questionnaire and module validity questionnaire. 

For the objectives-based study, respondents were given the pretest and motivation questionnaire to gauge their 
existing knowledge in Electrochemistry and their motivation level before they study Electrochemistry with 
IMMPA EC Lab. They were given 60 minutes to answer the pretest and 15 minutes to answer the motivation 
questionnaire. Then the respondents were gathered at the computer laboratory and briefed on how to use the 
IMMPA EC Lab. Respondents then started to explore the IMMPA EC Lab. Finally, they were given the post-test 
and motivation questionnaire. 

3. Results 

3.1 Quality of IMMPA EC Lab 

The quality of the module was determined by the module evaluation questionnaire. Similar questionnaires were 
given to both students and experts to determine their view on the quality of the IMMPA EC Lab. The students 
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rated IMMPA EC Lab as high quality with a mean of 3.94 (SD = 0.32) while experts rated it as very high quality 
with a mean of 4.45 (SD = 0.40). Means for all components in IMMPA EC Lab are shown in Table 8. 


Table 8. Mean scores for All Components in Module Evaluation Questionnaire 




Components 


Students 



Experts 




Mean 

SD 

Quality 

Mean 

SD 

Quality 

% 

Screen design 

3.95 

0.37 

High 

4.37 

0.35 

Very high 


A. 

Text 

4.04 

0.45 

High 

4.71 

0.30 

Very high 


B. 

Colour 

4.05 

0.48 

High 

4.25 

0.50 

Very high 


C. 

Graphic 

4.22 

0.47 

Very high 

4.43 

0.42 

Very high 


D. 

Animation 

4.15 

0.53 

High 

4.62 

0.41 

Very high 


E. 

Video 

4.04 

0.58 

High 

4.10 

0.42 

High 


F. 

Audio 

3.46 

0.57 

High 

4.18 

0.56 

High 

II. 

Navigation & Interactivity 

4.10 

0.40 

High 

4.59 

0.33 

Very high 

III. 

Language 

3.99 

0.49 

High 

4.65 

0.32 

Very high 

IV. 

Content 

4.21 

0.40 

Very high 

4.55 

0.45 

Very high 

V. 

Interest & Motivation 

3.77 

0.49 

High 

4.20 

0.59 

High 

VI. 

Pedagogical agent 







a) 

Professor T 

3.72 

0.59 

High 

4.21 

0.73 

Very high 


A. 

Facilitating Learning 

3.62 

0.70 

High 

4.31 

0.86 

Very high 


B. 

Credible 

4.06 

0.61 

High 

4.40 

0.49 

Very high 


C. 

Human-like 

3.58 

0.68 

High 

4.06 

0.85 

High 


D. 

Engaging 

3.71 

0.75 

High 

3.94 

0.75 

High 

b) 

Lisa 


3.84 

0.38 

High 

4.02 

0.72 

High 


A. 

Facilitating Learning 

3.90 

0.48 

High 

4.01 

0.66 

High 


B. 

Credible 

3.75 

0.50 

High 

4.26 

0.70 

Very high 


C. 

Human-like 

3.67 

0.51 

High 

3.86 

0.94 

High 


D. 

Engaging 

3.97 

0.53 

High 

3.97 

0.82 

High 

VII. 

Application of Theory 







a) 

Behaviourism Theory 

- 

- 

- 

4.65 

0.40 

Very high 

b) 

Cognitivism Theory 

- 

- 

- 

4.48 

0.50 

Very high 

c) 

Constructivism Theory 

- 

- 

- 

4.60 

0.45 

Very high 

d) 

Cognitive Load Theory 

- 

- 

- 

4.60 

0.35 

Very high 


A. 

Intrinsic cognitive load 

- 

- 

- 

4.65 

0.42 

Very high 


B. 

Extraneous cognitive load 

- 

- 

- 

4.61 

0.35 

Very high 



Split attention effect 

- 

- 

- 

4.54 

0.47 

Very high 



Worked-example effect 

- 

- 

- 

4.70 

0.47 

Very high 



Modality effect 

- 

- 

- 

4.83 

0.21 

Very high 



Redundancy effect 

- 

- 

- 

3.97 

0.81 

High 


C. 

Germane cognitive load 

- 

- 

- 

4.45 

0.39 

Very high 

e) 

Cognitive Theory of Multimedia 
Learning 

- 

- 

“ 

4.57 

0.31 

Very high 


A. 

Personalization principle 

- 

- 

- 

4.59 

0.29 

Very high 


B. 

Modality principle 

- 

- 

- 

4.79 

0.20 

Very high 


C. 

Spatial contiguity principle 

- 

- 

- 

4.65 

0.35 

Very high 


D. 

Temporal contiguity 

principle 

“ 

“ 

“ 

4.65 

0.44 

Very high 


E. 

Redundancy principle 

- 

- 

- 

3.89 

0.85 

High 


F. 

Voice principle 

- 

- 

- 

4.39 

0.59 

Very high 


G. 

Interactivity principle 

- 

- 

- 

4.73 

0.23 

Very high 

Total 



3.94 

0.32 

High 

4.45 

0.40 

Vey high 
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3.2 Module Reliability 

A module reliability questionnaire was administered to the students to determine the reliability of IMMPA EC 
Lab. The students rated the module as highly reliable with a mean of 3.87 (SD = 0.41). Means for all sessions in 
the IMMPA EC Lab are shown in Table 9. 


Table 9. Mean scores for All Sessions in IMMPA EC Lab 


Sessions in every Sub unit 

Mean 

SD 

Reliability 

Front page of sub units 

4.12 

0.54 

High 

Objectives 

3.66 

0.54 

High 

Think about it! 

3.76 

0.41 

High 

Do you still remember? 

3.81 

0.47 

High 

Give me your ideas... 

3.96 

0.60 

High 

Are you sure? 

3.97 

0.56 

High 

Let’s do it! !/Show time! 

3.80 

0.55 

High 

Practice makes perfect! 

3.91 

0.49 

High 

Before & after... 

4.04 

0.56 

High 

Test yourself! 

3.76 

0.52 

High 

Challenge yourself! 

3.81 

0.50 

High 


3.3 Module Validity 

A module validity questionnaire was administered to experts to evaluate the module in terms of validity. All the 
items in the questionnaire were based on the characteristics of module validity. The experts rated IMMPA EC 
Lab as a very highly valid module (M = 4.51, SD = 0.20). The mean value for each characteristic of module 
validity is summarized in Table 10. 


Table 10. Mean Scores for each Characteristic of Module Validity 


Characteristics of Module Validity 

Mean 

SD 

Validity 

Accurate target population 

4.66 

0.32 

Very high 

The teaching situation or implementation method is done very well 

4.60 

0.31 

Very high 

Enough time to implement the whole module 

4.14 

0.40 

High 

Potentially to increase students’ academic achievement 

4.66 

0.25 

Very high 

Potentially to change students’ behavior to be excellent in life 

4.51 

0.45 

Very high 

Total 

4.51 

0.20 

Very high 


3.4 Achievement Tests 

To carry out the objectives-based studies, achievement tests were given in the form of pretest and post-test 
before and after the intervention. The results of the pretest and post-test were used to examine the effect of 
IMMPA EC Lab on students’ understanding in Electrochemistry. Table 11 shows the t-test results for the pretest 
and post-test. 


Table 11. T-test table for Students’Achievement Tests 


Test 

N 

Mean 

Std. Deviation 

t value 

Sig (2-tailed) 

Post-test 

35 

26.98 

13.61 

8.97 

0.000* 

Pretest 

35 

9.42 

5.71 
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A paired-sample t-test was conducted to evaluate the impact of the IMMPA EC Lab on the students’ scores in the 
achievement test. There is a statistically significant increase in the achievement tests, from the pretest (M = 9.42, 
SD = 5.71) to the post-test [M = 26.98, SD = 13.61, t(34) = 8.97, p < 0.05]. The magnitude of the difference in 
the means is very large (eta squared = 0.5419) (Cohen, 1988). Hence, the first hypothesis is rejected which 
means that there is significant difference between the mean scores of the achievement test before and after the 
intervention. 

3.5 Motivation Questionnaire 

The motivation questionnaire was used to assess the students’ goals and value beliefs for Chemistry (especially 
Electrochemistry), their beliefs in their ability to succeed in the subject and their anxiety toward the test and 
examination on Electrochemistry. Table 12 shows the t-test results for the motivation questionnaire in the study. 


Table 12. T-test Table for Students’ Motivation Level 


Motivation 

N 

Mean 

Std. Deviation 

t value 

Sig (2-tailed) 

Post 

35 

3.68 

0.34 

2.42 

0.021* 

Pre 

35 

3.53 

0.32 




Data shows that there is a statistically significant increase in the level of motivation from pre-questionnaire (M = 
3.53, SD = 0.32) to the post-questionnaire [M = 3.68, SD = 0.34, t(34) = 2.42, p < 0.05]. The eta square is 0.079, 
which is considered to be a moderate effect size [41]. Hence, a second hypothesis is rejected; this implies that 
there is significant difference between the mean scores in motivation before and after intervention. 

4. Discussion 

The most important thing in a multimedia module is how the multimedia elements are used to express the 
content in a meaningful way and hence, to deliver it effectively to users. The multimedia elements used in 
IMMPA EC Lab were text, graphic, animation, simulation, video, audio and navigation. All these elements were 
evaluated in the module evaluation questionnaire. The students rated the highest mean for the graphic element 
among all the multimedia elements, while the experts’ highest mean is for the text element. The use of 
representational graphic (Levin, 1981) in IMMPA EC Lab made the information easier and more meaningful to 
the students. Text structures (Armbruster, 1986) and typographical signals made important texts more obvious. 
Consequently, students can identify the important part of the content and help them to prepare notes in the 
learning process. 

We found that students and experts were having different judgments regarding the PAs in IMMPA EC Lab. Lisa 
was more favored among the students but the experts preferred Professor T. Students rated Lisa as a better 
facilitator in learning, more human-like and more engaging as compared to Professor T. On the other hand, 
experts felt that Professor T is better at facilitating learning, more credible and more human-like compared to 
Lisa. Although both students and experts have different views about the PAs, both of them agreed that Professor 
T is more credible and Lisa is more engaging. Professor T as the expert in Electrochemistry always gives 
accurate information to the students in the learning process, so both students and experts rated him as more 
credible. On the other hand, Lisa as a learning companion, speaks in an energetic voice and always motivates 
and encourages the students, hence they rated her as more engaging. 

Application of theories gaining very high mean scores indicates that all the content delivery sequence, 
arrangement of text and graphic, design of the screen, use of multimedia elements, strategy, and techniques used 
in IMMPA EC Lab were following the principal and concepts in Behaviourism Theory, Cognitivism Theory, 
Constructivism Theory, Cognitive Load Theory and Cognitive Theory of Multimedia Learning. However, the 
mean for redundancy effect and redundancy principal is quite low compared to means for other effects in 
Cognitive Load Theory and principals in Cognitive Theory of Multimedia Learning. Experts felt that texts 
displayed on the screen when the narration is given are redundant (Craig, Driscoll, & Gholson, 2004; Craig et al., 
2002; Sweller & Chandler, 1994) to the learners because the information is the same. Students’ working memory 
was loaded because they had to read the text and listen to the narration at the same time. The language used in 
IMMPA EC Lab is English, following the Teaching and Learning of Science and Mathematics using English 
Project (Pengajaran dan Pembelajaran Sains dan Matematik dalam Bahasa Inggeris, PPSMI) implemented by the 
government since 2003. Comprehension of the decoded message is affected by the ability of the receiver to 
comprehend the message (Heinich, Molenda, Russel, & Smaldino, 2002). Since English is not our mother tongue, 
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it would create problems in delivering information from PAs to students if only audio communication is involved. 
Hence, on some of the screens, we have decided to deliver the information in both audio and visual forms. 

High mean scores for every part in IMMPA EC Lab indicated that the module is reliable to be used as a learning 
module in Electrochemistry. The items in the module reliability questionnaire were developed according to the 
activities in IMMPA EC Lab. High mean scores for the module reliability showed that the students were able to 
follow all the steps and activities in IMMPA EC Lab successfully (Russell, 1974). 

The overall mean for module validity is very high indicating that all the experts agreed that the IMMPA EC Lab 
is a valid learning module according to the characteristics of module validity suggested by Russell (1974). 
Majority of the experts agreed that the content of IMMPA EC Lab is following the Chemistry syllabus and the 
students can fulfill all the objectives through the activities designed. The use of animations and videos helped the 
students in understanding the concepts of Electrochemistry especially at microscopic level. 

Students showed improvement in the achievement test, indicating that they gained knowledge about 
Electrochemistry after learning with IMMPA EC Lab. This showed that IMMPA EC Lab is effective in 
increasing students’ knowledge. However, we found that students are still weak in certain Electrochemistry 
concepts especially at microscopic and symbolic levels. For instance, the students still could not differentiate 
between the flow of electrons in the conductors and the flow of ions in the electrolytes (Kamisah & Lee, 2012; 
Lee & Mohammad Yusof, 2009; Lee, 2008; Ozkaya, Uce & §ahin, 2003; Sanger & Greenbowe, 1997a). Besides 
that, we found that students always assumed that anions would be attracted to the cathode while cations would be 
attracted to the anode (Kamisah & Lee, 2012; Lee & Mohammad Yusof, 2009; Lee, 2008). This is because 
anions are negative ions and the students in this study assumed that negative ions would be attracted to negative 
electrode and vice versa. However, students’ results showed improvement from pretest to post-test, indicating 
that the animations in the Micro-World had helped the students in understanding Electrochemistry concepts 
microscopically. The use of multimedia creates the environment where students could visualize the abstract 
chemical processes via animations and videos at macroscopic, microscopic and symbolic levels (Bowen, 1998; 
Burke et al. 1998; Rodrigues et al. 2001; Russell et al. 1997). 

Overall, the students’ motivation level increased for each construct in the motivation questionnaire except for the 
test anxiety construct which maintained the same. The mean value for self-efficacy for learning and performance 
construct showed the biggest increase from 3.30 (SD = 0.42) to 3.54 (SD = 0.50) after the students had studied 
with the IMMPA EC Lab. Self-efficacy refers to personal beliefs about having the means to learn or perform 
effectively (Zimmerman, 2000). High self-efficacy beliefs enabled the students to be more motivated to learn 
Electrochemistry and hence, their test anxiety level was low (M = 2.06, SD = 0.67). High self-efficacy values are 
related to relatively high intrinsic motivation values. In this study, the students’ intrinsic goal orientation value is 
3.84 (SD = 0.48), showing that the students enjoyed learning with the IMMPA EC Lab. The variety of feedback 
given by the PAs and the videos shown during the discussions attracted the students’ attention to study the topic. 
The students showed the highest extrinsic goal orientation (M = 4.54, SD = 0.40) among all the constructs, 
indicating that they were trying to show to others that they could perform well in Chemistry. 

We considered the suggestions and recommendations by students and experts, and made some corrections and 
improvements accordingly. For instance, some of the important keywords in the answer scheme were highlighted 
so that students will take note of them. The button for printing function was added to some of the screens so that 
users can print out related pages. One of the experts mentioned about adding the concentration cell in IMMPA 
EC Lab, but we did not consider it because the concentration cell is not in the syllabus for Form Four Chemistry. 
Some students complained that the background music was annoying and affecting their concentration during the 
learning process. The background music is actually interactive; users can turn it off if they do not wish to listen. 
We do not plan to get rid of the background music as some students may have strong musical intelligence and 
need music in the learning process. 

5. Conclusion 

The formative evaluation has been carried out in the form of objectives-based study and connoisseur-based study 
approaches involving some students and experts. Students’ acquisition of knowledge and motivation levels were 
tested and measured via the achievement tests and motivation questionnaire. Results showed statistically 
significant increment of test scores and motivation mean scores after the intervention. The objectives-based 
study has proven that the IMMPA EC Lab developed is effective in raising students’ knowledge and motivation 
level in the learning of Electrochemistry. This is parallel with studies abroad (Johnson et al., 2000; Kizilkaya & 
Askar, 2008; Moundridou & Virvou, 2002; Moreno, Mayer & Lester, 2000) where students were found to be 
more motivated and interested, and achieved higher performance when learning with tutorials supported by PAs. 
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On the other hand, the connoisseur-based study involving some experts in Electrochemistry and instructional 
design was carried out to evaluate the quality of the module developed. Experts rated IMMPA EC Lab as of very 
high quality and the content is valid. Besides that, results from students also showed that the module is reliable 
and they managed to carry out the activities in the module. 

Suggestions and recommendations from respondents were considered and some corrections have been made to 
improve the quality of IMMPA EC Lab. We hope that IMMPA EC Lab can meet requirements for students from 
different backgrounds and become their learning aid in the learning of Electrochemistry. Field trials in the form 
of summative evaluation would be carried out to evaluate the improved IMMPA EC Lab. The purpose of 
summative evaluation is to assess the actual implementation of IMMPA EC Lab and study its effectiveness in 
improving students’ knowledge and motivation in learning Electrochemistry. The number of respondents should 
be increased and another study would be carried out using quasi-experimental design to compare the outcomes 
between control and treatment groups. It is hoped that the improved IMMPA EC Lab is able to increase students’ 
knowledge and enhance their motivation in the learning of Electrochemistry. 
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